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1 Abstract 



We introduce an extension of nonparametric DS inference for arbitrary uni- 
variate CDFs to the case in which some failure times are (right)-censored, 
and then apply this to the problem of assessing evidence regarding assertions 
about relative risks across two populations. The approach enables explo- 
ration of the sensitivity of survival analyses to assumed independence of the 
missing data process and the failure proces. We present an application to the 
partially efficacious RV144 (HIV-1) vaccine trial, and show that the strength 
of conclusions of vaccine efficacy depend on assumptions about the maximum 
failure rates of the subjects lost-to-foUowup. 

2 Introduction 

Given data from an HIV vaccine trial, we are interested in determining 
whether the rate of HIV acquisition is lower among vaccine recipients (r„) 
than among placebo recipients (r^) (over a given period of time-since- vaccination, 
eg within the first 18 months). We arc furthermore interested in estimat- 
ing the ratio of these rates (with a ratio of r^/rp < 1 indicating a lower 
rate among vaccine recipients). Finally, we define vaccine efficacy (VE) as 
1 — r^/rp, and wish to test assertions about this quantity, such as "VE > 
50%". 

The data is of the form of right-censored survival data. That is, we have 
a total number of subjects rrip assigned to the placebo group and a total 
number assigned to the vaccine group. For each subject i we have an 
estimated infection time ti if subject i became infected during the course 
of the trial, or a loss-to-foUowup time k if he did not become infected (this 
might be the total trial duration for most participants, but could also be 
some earlier time if the subject dropped out of the trial). We also know 
the vaccine treatment assignment indicator yi for each subject, such that 
Ei Vi = my and Ei (1 - Vi) = rUp. 

Wishing to avoid parametric assumptions, we temporarily eschew the 
usual Cox regression approach, and introduce instead a nonparametric method 
employing Dempster-Shafer (DS) analysis. Unlike the alternative nonpara- 
metric Kaplan-Meier method, in which right-censored data are accomodated 
by conditioning on non-censoredness (or "at risk"-ness), this approach con- 
siders the whole dataset for its inferences. As such its conclusions are not 
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subject to the post-randomization selection bias concerns which hmit causal 

interpretability of estimates based on Kaplan-Meier curves. 

Like the Kaplan-Meier approach, this DS method for estimating hazard 
ratios is based on estimates of the cumulative distribution functions (CDFs) 
of the "failure times" (here infection times) within each treatment group. If 
we knew the (true, population) CDFs F„ and Fp for these groups, we could 
determine the failure rates r.^ and Vp for any arbitrary time interval (ti.tu) 
by eg = F^itu) — F^{ti). This is just the population fraction that fail in 
that time interval. 

To describe the method we therefore begin with the estimation of a single 
CDF. Thereafter we will consider how two such estimates may be used in 

concert to make inferences about a ratio of two rates such as r„/rp. We note 
throughout that estimates of CDFs have many potential uses, of which the 
present discussion will focus on just this one (which has relevance to the 
analysis of survival data such as the vaccine trial data which we use as a 
concrete illustration) . 

3 Dempster-Shafer Analysis 

Dempster-Shafer analysis (DSA) is an approach to statistical reasoning which 
tolerates evidence that is ambiguous with respect to an assertion of interest. 
For example, using DSA to address the hypothesis that a possibly-unfair coin 
has probabihty p — .5 of heads after observing 3 heads in 10 tosses yields 
evidence of the form (P, Q, i?), where P is evidence "for" the hypothesis that 
p = .5, Q is evidence "against" it, and the remainder, R = 1 — {P + R), is 
evidence that is ambiguous with respect to the hypothsis (we say it is the 
probability of "don't know"). 

[etc .. copied and perhaps altered from the DSBanff paper or elsewhere] 

4 Nonparametric DS Estimation of a CDF 

We now turn to the nonparametric DS estimation of a univariate, invertable 
CDF F from data assumed to have been randomly sampled from it. Suppose 
we observe m data points and order them such that Xi < X2 < ■ ■ ■ < Xm- We 
make inferences about F via the auxiliary random variables Yi, ..,Yjn, which 
are independent and Uniformly distributed. The intuition is as follows: if 
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we knew the inverse function F~^, we could have generated the data by first 
drawing m uniforms and then applying to each the inverse CDF, such that 
for any y we define a corresponding x = F^^{y). On an x-y plot of the CDF 
F, we could draw a horizontal line from each to the F curve, and from 
there drop vertically to the x axis to determine the value of Xj. 

We needn't assume that the data were in fact generated in this way, but 
this perspective allows us to make inferences about F . We define random 
variables Kj for the height of the CDF at each observed value Xi. Since the 
Xi are ordered, the correspondingly ordered Fj are jointly distributed as the 
order statistics of m uniforms. 

Evidence for and against assertions about the function F may be assessed 
via the random variables Y . For instance if we observe 10 ordered failure 
times Xi, a;2, . . . , a;io, and we wish to infer the population fraction whose 
failure times fall between the fourth and sixth of these (that is, we wish 
to infer F~^{xq) — F~^(a;4), we may do so via the joint distribution of the 
corresonding random variables Yq and I4 (as Yq — I4). That difference is 
distributed as {Yq - Y^) ~ Beta(2, 9). 

DSA allows us to make inferences about the CDF at points other than 
the observed values, since we are able to represent our uncertainty using 
a mass function over sets (or intervals) of values. For instance, if the val- 
ues of Xi, X2-1 2^3, anda:4 (of 10 observed failure times) were 10, 30, 55, andlOO, 
respectively, and we were interested in testing the hypothesis that at least 
15% of the population fails between times 25 and 75, we could accumu- 
late evidence for this assertion as the probability that (Y3 — Y2) > 15% , 
and against it by the probabihty that (F4 — Yi) < 15%. Such evidence 
will not generally sum to 1; the remaining evidence is ambiguous and con- 
tributes to the residual "don't know" . In this instance, we get a (P, Q, R) of 
(P= (l-p6eta(.15, 1,10),Q =p6eta(.15,3,8),l-(P + Q)) = (0.20, .18, .62). 
Note that these events are mutually exclusive and thus may be computed 
separately. 

If we introduce right-censored data of the "loss to foUowup" (LTF) type, 
we simply treat the number of failures by a particular time (and hence the 
order-statistic-of-interest) as somewhere between the observed cumulative 
number of failures and that number plus the cumulative number of LTFs by 
that time. For instance if we modify the above example by converting one 
of the previously-observed-to-be-larger values (eg Xiq) by a subject that is 
lost to followup at time 50, then we aren't sure whether the order-indices 
of "X3 = 55" and "X4 = 100" are (3,4), (3,5), or (4,5). This increases 
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R, because now we calculate the evidence Q against the hypothesis as the 
probability that Y^—Yi < 15%, because wc don't know whether X4 is above or 
below the failure time of the lost-to-followup subject. Thus we get a (P, Q, R) 
of(P= (l-pbeta(.15, l,10),Q = pbeta(.15,4, 7),l-(P+g)) = (.2, .05, .75). 

Note that we know that the number of new failures between any two time 
points is bounded below by the observed number of new failures occuring 
after the lower time point and up to-and-including the upper time point. It 
is bounded above by that number plus the total number of LTFs that have 
accumulated up to the upper time point. If we represent the data as a two- 
row matrix C with cumulative sums of failures Ci^k and cumulative sums of 
LTFs C2,jfc at each discrete time point tk for which we have any observations, 
then the number of new failures between any two of these time points tj < tk 
is bounded below by dj^k '■= Ci,k — Cij and above by ej^k '■= dj^k + C2,k- 

Returning to our initial example (with no LTFs), suppose we wish to test 
the hypothesis that between 10% and 20% of the population fails between 
times 25 and 75. The evidence for this assertion is the probability of the event 
that both {Y3 - ¥2) > 10% AND (^4 - Yi) < 20%. The evidence against 
the assertion is given by the probability that either {Y4 — Yi) < 10% OR 
(Is — Y2) > 20%. In general, to test a hypothesis that between (quantiles) 
qi and g„ of the population falls between (times) ti and we first need to 
identify the indices into the columns of matrix C of the nearest observed time 
points to ti and t„, both above {k^ and ) and below {k'l and /c^ ) each. In 
our example, U = 25, U = 75, k^ = 1, k^^ = 2, A;^*^ = 3, k^J= 4. 

With these in hand, the general formula, in the absence of LTFs, is that 
the evidence for the hypothesis (that between qi and q^ of the population 
fails between times ti and ty) is given by the probability of the event that 
both Yc^ . - .a > Qi AND Yc^ - Yn . < qu- The evidence against 

it is given by the sum of the probabilities that Yq^ — Yc t < and 

that kl — Yc^ ^„ > qu- Due to the symmetry of the intervals separating the 

uniform order statistics Y, the actual orders are irrelevant to these probability 
calculations: the differences are sufficient. We will refer to the difference 
Ci ub — Ci as the "internal interval count" f „ and to the difference Ci k"- — 
Cii^b as the "external interval count" Vx- What matters is the probability 
distributions of intervals Wn and of size f„ and v^, which marginally 
are given by Beta distributions: Wn ~ Beta.{vn,m + 1 — Vn) and ~ 
Beta(i'^,m + 1 — v^), where m is the total number of subjects. In words. 
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P, the evidence for the hypothesis, is given by the probabihty that both 
the internal interval Wn is greater than the lower quantile qi and that the 
external interval Wx is less than the higher quantile Qu- Q, the evidence 
against the hypothesis, is given by the probability that either the internal 
interval is too large or that the external interval is too small. The remaining 
evidence is ambiguous with respect to the hypothesis, so it is assigned to R, 
"don't know". 

In the presence of LTFs, we must consider that the actual number of fail- 
ures between two time points tj < tk is potentially unknown, but is bounded 
(by dj^k and Cj^k as described above). Thus where in the above formulae we 
used the number of failures directly from the C matrix, in general we must 
use the appropriate upper or lower bound. If we define the "maximum inter- 
nal interval count" as e^a and the "minimum internal interval count" 

as d^a , and likewise define the "maximum external interval count" v'^ 

tl ' tu 

as 61.6 ua and the "minimum external interval count" vL as di.b ua , then we 
can in general compute the evidence Q against the hypothesis as the sum 
of the probablities that the maximum internal interval is less than the 
lower quantile qi and that the minimum external interval is greater than 
the upper quantile Qu- 

We compute the evidence P for the hypothesis as the probability that 
(simultaneously) both the minimum internal interval is greater than the 
lower quantile qi and the maximum external interval is less than the 
upper quantile qu- This is most readily calculated via its complement, since 
this is just 1 — {P{Wl < qi) + P{W^ > qu))- Since these are mutually 
exclusive events, the probabilities may be computed separately, and each is 
computed simply via the Beta CDF. 
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Conditioning on uniforms to evaiuate the hypothesis 
that >= 15% of the popuiation faiis in the time interval (30, 100). 
In this exampie, the evidence supports the hypothesis. 



Conditioning on uniforms to evaluate the hypothesis 
that >= 15% of the population fails in the time intervai (30, 100). 
in this exampie, the evidence is against the hypothesis. 
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(a) Example of uniform draws that support the (b) Example of uniform draws that contradict 
hypothesis. the hypothesis. 



Figure 1: Examples of conditioning on uniform samples to make nonpara- 
metric inferences about a CDF in the absence of missing data: evidence for 
and against |(b)| the hypothesis that at least 15% of the population fails 



in the time interval (30, 100). Note that because the times 30 and 100 are 
observed, the evidence is unambiguous: the lower and upper bounds (on the 
population fraction that fail in that interval) coincide. 
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Conditioning on uniforms to evaluate the assertion 
that >= 15% of the population fails in the time intervai (25, 75). 
in this example, the evidence supports the assertion. 
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Conditioning on uniforms to evaluate the assertion 
that >= 15% of the population fails in the time interval (25, 75). 
In this example, the evidence is against the assertion. 
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(a) Example of uniform draws that support the (b) Example of uniform draws that contradict 
hypothesis. the hypothesis. 



Conditioning on uniforms to evaluate ttie assertion 
tliat >= 15% of tile population fails in the time interval (25, 75). 
In this example, the evidence is ambiguous. 
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(c) Example of uniform draws that are ambigu- 
ous with respect to the hypothesis. 



Figure 2: Examples of conditioning on uniform samples to make nonpara- 
metric inferences about a CDF in the absence of missing data: evidence for 
(a) , against |(b)[ and ambiguous (c) with respect to the hypothesis that at 
least 15% of the population fails in the time interval (25, 75). Note that 
because these times (25 and 75) fall between observed times, there is am- 
biguity about how many failures occigr, leading to a difference between the 
lower bound and the upper bound on the population fraction failing in that 
interval, which is reflected in the possibility of evidence that is ambiguous 
with respect to the hypothesis. 



Conditioning on uniforms to evaluate the assertion 
that >= 15% of the population fails in the time interval (30, 100). 
In this example, the evidence supports the assertion. 
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Conditioning on uniforms to evaluate the assertion 
that >= 15% of the population fails in the time interval (30, 100). 
In this example, the evidence is against the assertion. 
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(a) Example of uniform draws that support the (b) Example of uniform draws that contradict 
hypothesis. the hypothesis. 



Conditioning on uniforms to evaluate ttie assertion 
that >= 15% of file population fails In the time Interval (30, 100). 
In this example, the evidence is ambiguous. 
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(c) Example of uniform draws that are ambigu- 
ous with respect to the hypothesis. 



Figure 3: Examples of conditioning on uniform samples to make nonpara- 
metric inferences about a CDF in the absence of missing data: evidence for 
(a) , against |(b)[ and ambiguous (c) with respect to the hypothesis that at 
least 15% of the population fails in the time interval (30, 100). Here, ambi- 
guity is introduced by the subject that was lost-to-foUowup at time 20, who 
may or may not have failed in the interval (30, 100). 



Conditioning on uniforms to evaiuate the hypothesis 
that >= 15% of the population fails in the time intervai (25, 75). 
In this exampie, the evidence supports the hypothesis. 
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Conditioning on uniforms to evaluate the hypothesis 
that >= 15% of the popuiation faiis in the time interval (25, 75). 
in this exampie, the evidence is against the hypothesis. 
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(a) Example of uniform draws that support the (b) Example of uniform draws that contradict 
hypothesis. the hypothesis. 



Conditioning on uniforms to evaiuate the hypothesis 
that >= 15% of the population fails in the time intervai (25, 75). 
In this example, the evidence is ambiguous. 
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(c) Example of uniform draws that are ambigu- 
ous with respect to the hypothesis. 



Figure 4: Examples of conditioning on uniform samples to make nonpara- 
metric inferences about a CDF in the absence of missing data: evidence for 
(a) , against |(b)[ and ambiguous (c) with respect to the hypothesis that at 
least 15% of the population fails in the time interval (25, 75). Here, ambi- 
guity is introduced both by the subject that was lost-to-followup at time 20, 
who may or may not have failed in tj^ interval (25, 75), and because these 
times (25 and 75) fall between observed times. 



